Validate converted HTTP/2 headers and trailers#2275
Conversation
HTTP/2 decoding already validates response header names. Rechecking them while converting to HttpHeaders repeats per-character work for every response and trailer. Use a factory that disables only name validation while preserving value validation, including CR/LF rejection. Cover both properties with focused tests. Codex on behalf of Pavel Ptashyts Co-Authored-By: Codex <[email protected]>
| public final class Http2Handler extends AsyncHttpClientHandler { | ||
|
|
||
| private static final HttpVersion HTTP_2 = new HttpVersion("HTTP", 2, 0, true); | ||
| private static final HttpHeadersFactory RESPONSE_HEADERS_FACTORY = |
There was a problem hiding this comment.
This is the core issue. Turning off name validation here is not removing a duplicate check, it is removing the only check. Netty's HTTP/2 name validation rejects uppercase and nothing else, so everything the HTTP/1 token validator was catching now passes straight through into the headers we hand to user code.
I suggest dropping withNameValidation(false) and using DefaultHttpHeadersFactory.headersFactory() as is, or just going back to new DefaultHttpHeaders(). Keeping the factory as a hoisted static is fine and thread safe under Sharable, so that part can stay if you prefer it.
There was a problem hiding this comment.
Agreed. Removed withNameValidation(false). Converted response headers now use Netty's default name and value validators.
| } | ||
|
|
||
| static HttpHeaders copyHttp2Headers(Http2Headers h2Headers) { | ||
| // The HTTP/2 decoder validates names but not values by default. Avoid repeating the name scan while |
There was a problem hiding this comment.
This comment is the part I would most want fixed even after the code is sorted out. The second clause is correct, the decoder genuinely does not validate values. The first clause is not, and left in the tree it will convince the next person reading this method that the copy is safe when it is not. Comments that assert an invariant tend to outlive the reasoning behind them, so it is worth rewriting rather than deleting.
There was a problem hiding this comment.
Rewrote the comment to state the actual boundary: HTTP/2 validation does not enforce complete HTTP token syntax or decoded values, so both are validated during conversion.
| trailingHeaders.add(name, entry.getValue()); | ||
| } | ||
| }); | ||
| HttpHeaders trailingHeaders = copyHttp2Headers(h2Headers); |
There was a problem hiding this comment.
Not introduced by this PR, but this is the natural moment to raise it since you are centralising the code. The trailers path uses the same response factory, so it never gets Netty's trailer specific restrictions. trailersFactory() additionally rejects Content-Length, Transfer-Encoding and Trailer in trailers, per RFC 9110 section 6.5.1, and I confirmed those three are rejected by that factory and accepted by the one used here.
Worth knowing that the H2 layer does not cover this either. I checked HttpHeaderValidationUtil.isConnectionHeader and it returns true for transfer-encoding, connection, te, keep-alive and upgrade, but false for content-length. So nothing anywhere currently rejects Content-Length in an HTTP/2 trailer. Happy for this to be a follow up rather than scope creep here, but note that trailersFactory().withNameValidation(false) would not help, since that flag disables the trailer restrictions too. The flag has to go first.
There was a problem hiding this comment.
Applied the trailer-specific factory in this revision. Content-Length, Transfer-Encoding, and Trailer are now rejected before trailing headers reach the handler.
| // The HTTP/2 decoder validates names but not values by default. Avoid repeating the name scan while | ||
| // retaining the HTTP/1 value validator that rejects CR/LF and other prohibited characters. | ||
| HttpHeaders headers = RESPONSE_HEADERS_FACTORY.newHeaders(); | ||
| h2Headers.forEach(entry -> { |
There was a problem hiding this comment.
Small aside while this loop is being touched. The lambda captures headers, so it cannot be cached and a fresh instance is allocated per response and again per trailer frame. A plain enhanced for loop over the entries removes that allocation at no cost to safety, which is a cleaner win than the one this PR is going for.
There was a problem hiding this comment.
Replaced the capturing lambda with an enhanced for loop shared by response headers and trailers.
| HttpHeaders headers = RESPONSE_HEADERS_FACTORY.newHeaders(); | ||
| h2Headers.forEach(entry -> { | ||
| CharSequence name = entry.getKey(); | ||
| if (name.length() > 0 && name.charAt(0) != ':') { |
There was a problem hiding this comment.
Worth being aware that this guard is now the only thing stopping empty names. Previously the default validator rejected them as well. With name validation off the replacement is a not null check, so an empty name would be accepted if it ever reached the add call. Behaviour is unchanged today because this line filters them, but the backstop is gone and nothing tests this line.
There was a problem hiding this comment.
Added a focused test for the empty-name guard. Empty names are skipped before insertion, while the destination validator remains the backstop for malformed non-empty names.
| class Http2HandlerTest { | ||
|
|
||
| @Test | ||
| void copiesDecoderValidatedHeaderNamesWithoutRevalidation() { |
There was a problem hiding this comment.
This test pins the regression as intended behaviour, and the method name states the claim that does not hold. The name used here is not decoder validated. Passing false to DefaultHttp2Headers looks like it is there to construct something the decoder would have rejected, but DefaultHttp2Headers with validation on accepts this name too, because it contains no uppercase. A space is in the forbidden range in RFC 9113 section 8.2.1.
I suggest inverting it, so it asserts that a name containing CRLF is rejected. That is the property worth locking down, and it is the one that regressed. If it helps I have a reproduction that goes through the real encoder and decoder rather than constructing headers by hand, and I am happy to contribute it.
There was a problem hiding this comment.
Agreed and inverted the test. It now verifies that a CRLF-containing name is rejected during conversion; the previous invalid-name acceptance claim has been removed.
| } | ||
|
|
||
| @Test | ||
| void stillValidatesHeaderValues() { |
There was a problem hiding this comment.
This one is good and I would keep it regardless of what happens to the rest of the PR. Value validation on this path was not covered before and it is the property that actually needs a guard, since the decoder does not check values at all.
There was a problem hiding this comment.
Kept and renamed the value-validation test. It continues to verify that CRLF in a decoded header value is rejected.
Restore complete HTTP token and value validation when converting decoded HTTP/2 headers. Use Netty's trailer-specific factory so prohibited trailing fields cannot reach handlers. Replace capturing copy lambdas with enhanced loops and cover malformed names, values, empty names, and prohibited trailers. Codex on behalf of Pavel Ptashyts Co-Authored-By: Codex <[email protected]>
|
Addressed all review feedback in 0411239:
The focused tests and the full JDK 11 ./mvnw clean verify pass. |
Summary
Motivation
The original implementation assumed the HTTP/2 decoder had already performed complete header-name validation. Netty 4.2.15 validates HTTP/2 casing and pseudo-header rules, but it does not enforce the complete HTTP token syntax, and decoded values are not validated by default. Disabling name validation therefore removed the only complete check before headers reached user code.
This revision keeps Netty's normal response-header validation and uses its trailer factory for Content-Length, Transfer-Encoding, and Trailer restrictions. The shared enhanced-for copy path still avoids a capturing lambda allocation for each response and trailer frame.
Validation
Attribution
Codex on behalf of Pavel Ptashyts